Spoken Dialogue Management Using Hierarchical Reinforcement Learning and Dialogue Simulation
نویسنده
چکیده
Speech-based human-computer interaction faces several difficult challenges in order to be more widely accepted. One of the challenges in spoken dialogue management is to control the dialogue flow (dialogue strategy) in an efficient and natural way. Dialogue strategies designed by humans are prone to errors, labour-intensive and non-portable, making automatic design an attractive alternative. Previous work proposed addressing the dialogue strategy design as an optimization problem using the reinforcement learning framework. However, the size of the state space grows exponentially according to the state variables taken into account, making the task of learning dialogue strategies for large-scale spoken dialogue systems difficult. In addition, learning dialogue strategies from real users is a very expensive and time-consuming process, making automatic learning an attractive alternative. To address these research problems three lines of investigation are proposed. Firstly, to investigate a method to simulate task-oriented human-computer dialogues at the intention level in order to design the dialogue strategy automatically. Secondly, to investigate a metric to evaluate the realism of simulated dialogues. Thirdly, to make a comparative study between hierarchical reinforcement learning methods and reinforcement learning with function approximation, in order to find an effective and efficient method to learn optimal dialogue strategies in large state spaces. Finally, a timeline for the completion of this research is proposed.
منابع مشابه
Hierarchical Reinforcement Learning for Spoken Dialogue Systems
This thesis focuses on the problem of scalable optimization of dialogue behaviour in speech-based conversational systems using reinforcement learning. Most previous investigations in dialogue strategy learning have proposed flat reinforcement learning methods, which are more suitable for small-scale spoken dialogue systems. This research formulates the problem in terms of Semi-Markov Decision P...
متن کاملEvaluation of a hierarchical reinforcement learning spoken dialogue system
We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the ...
متن کاملOn-Line Learning of a Persian Spoken Dialogue System Using Real Training Data
The first spoken dialogue system developed for the Persian language is introduced. This is a ticket reservation system with Persian ASR and NLU modules. The focus of the paper is on learning the dialogue management module. In this work, real on-line training data are used during the learning process. For on-line learning, the effect of the variations of discount factor (g) on the learning speed...
متن کاملOn-Line Learning of a Persian Spoken Dialogue System Using Real Training Data
The first spoken dialogue system developed for the Persian language is introduced. This is a ticket reservation system with Persian ASR and NLU modules. The focus of the paper is on learning the dialogue management module. In this work, real on-line training data are used during the learning process. For on-line learning, the effect of the variations of discount factor (g) on the learning speed...
متن کاملHierarchical Reinforcement Learning of Dialogue Policies in a development environment for dialogue systems: REALL-DUDE
We demonstrate the REALL-DUDE system1, which is a combination of REALL, an environment for Hierarchical Reinforcement Learning, and DUDE, a development environment for “Information State Update” dialogue systems (Lemon and Liu, 2006) which allows non-expert developers to produce complete spoken dialogue systems based only on a Business Process Model (BPM) and SQL database describing their appli...
متن کامل